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Abstract. Online dating sites have become popular platforms for people 
to look for potential romantic partners. Different from traditional user- 
item recommendations where the goal is to match items (e.g., books, 
videos, etc) with a user’s interests, a recommendation system for online 
dating aims to match people who are mutually interested in and likely 
to communicate with each other. We introduce similarity measures that 
capture the unique features and characteristics of the online dating net¬ 
work, for example, the interest similarity between two users if they send 
messages to same users, and attractiveness similarity if they receive mes¬ 
sages from same users. A reciprocal score that measures the compatibility 
between a user and each potential dating candidate is computed and the 
recommendation list is generated to include users with top scores. The 
performance of our proposed recommendation system is evaluated on a 
real-world dataset from a major online dating site in China. The results 
show that our recommendation algorithms significantly outperform pre¬ 
viously proposed approaches, and the collaborative filtering-based algo¬ 
rithms achieve much better performance than content-based algorithms 
in both precision and recall. Our results also reveal interesting behavioral 
difference between male and female users when it comes to looking for 
potential dates. In particular, males tend to be focused on their own in¬ 
terest and oblivious towards their attractiveness to potential dates, while 
females are more conscientious to their own attractiveness to the other 
side of the line. 


1 Introduction 

Online dating sites have become popular platforms for people to look for poten¬ 
tial romantic partners, offering an unprecedented level of access to possible dates 
that is otherwise not available through traditional means. According to a recent 
survejH, 40 million single people (out of 54 million) in the US have signed up 
with various online dating sites such as Match.com, eHarmony, etc, and around 
20% of currently committed romantic relationships began online, which is more 
than through any means other than meeting through friends. 

® http://statisticbrain.com/online-dating-statistics 
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Many online dating sites provide suggestions on compatible partners based 
on their proprietary matching algorithms. Unlike in traditional user-item recom¬ 
mendation systems where the goal is typically to predict a user’s opinion towards 
passive items (e.g., books, movies, etc), when making recommendation of poten¬ 
tial dates to a user (referred to as service user) on an online dating site, it is 
important that not only the recommended users match the user’s dating inter¬ 
est, but also the recommended users are interested in the service user and thus 
likely to reciprocate when contacted. A successful online dating recommendation 
system should match users with mutual interest in each other and hence result 
in better chances of interactions between them and improved user satisfaction 
level. 

In this paper, we study the reciprocal online dating recommendation system 
based on a large real-world dataset obtained from a major online dating site in 
China with a total number of 60 million registered users. In particular, given a 
user, we seek to identify a set of users who are most likely to be contacted by the 
service user when recommended and reciprocate when contacted. 

The characteristics of the our online dating network present unique oppor¬ 
tunities and challenges to the reciprocal recommendation problem. First, there 
is a rich set of user attributes available in our dataset that can be used in 
the prediction models. These include a user’s age, gender, height, weight, ed¬ 
ucation level, income level, house ownership, geographic location, occupation, 
interests/hobbies, number of photos, etc. In addition, there are a variety of on¬ 
line dating specific information including a user’s preference in potential dates 
(age range, height range, education level, income range, geography location, etc), 
and his/her dating and marriage plan (when to get married, whether to live with 
parents and have child after marriage, marriage ceremony style, etc). Moreover, 
our dataset contains the communication traces between users, i.e., who sent or 
replied to messages to whom and the associated timestamps. As shown in our 
earlier paper [1], the communication trace of a user reflects his/her actual dating 
preference, which may significantly deviate from his/her stated preference and 
thus should play an important role in the design of the recommendation system. 

Due to the heterogeneous dating nature in our recommendation problem 
(dating is restricted to users of opposite genders on the online dating site in 
our study), previous approaches designed for friend recommendation in conven¬ 
tional online social networks such as Facebook and Linkedin cannot be directly 
applied. For example, the number of common neighbors is often used for friend 
recommendation for conventional social networks, i.e., the more common friends 
two users share, the more likely they will become friends and thus should be 
recommended to each other. On a heterosexual online dating site, however, a 
user is only interested in contacting other users of opposite gender, resulting in 
a bipartite network between males and females. There is no common neighbors 
between a service user and recommended users since they are of different gen¬ 
ders. To this end, we will need to devise appropriate mechanisms that accounts 
for the special characteristics of the online dating network. 

The contributions of this paper are summarized as follows. 
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— We present a recommendation system that aims to match users who are most 
likely to communicate with each other. We introduce similarity measures that 
capture the unique features and characteristics of the heterogeneous online 
dating network. In particular, we build a preference model for each service 
user based on the attributes of users who have been contacted by the service 
user. We also characterize the interest similarity between two users if they 
send messages to same users, and attractiveness similarity if they receive 
messages from same users. Based on these similarity measures we compute 
the compatibility between a service user and potential dating candidates and 
the recommendation list is generated to include candidates with top scores. 
Using a combination of similarity measures, we construct a set of content- 
based and collaborative filtering-based algorithms with different measures of 
compatibility between users. 

— We evaluate the performance of our proposed algorithms on the real-world 
online dating dataset that consists of 200,000 users and around two mil¬ 
lion messages over a period of two months. Our results show that both 
content-based and collaborative filtering-based recommendation algorithms 
presented in our paper significantly outperform previously proposed ap¬ 
proaches. Also, compared to the content-based algorithms, our collaborative 
filtering-based algorithms achieve much better performance in both preci¬ 
sion and recall. Moreover, most of our proposed recommendation algorithms 
place the relevant recommendations (users who have been actually contacted 
by and replied to the service user) in the top 30% to 50% positions of the 
recommendation list. This is an important performance measure as users 
tend to look at the list from top to bottom. 

— Our results also reveal interesting behavioral difference between male and 
female users when it comes to looking for potential dates. Among the col¬ 
laborative filtering-based algorithms, the best performance for male users is 
achieved when the recommender captures the attractiveness of recommended 
users to the service user and interest from the service user in recommended 
users. On the contrary, the best performance for females is achieved when 
recommended users are interested in the service user and the service user 
is attractive to recommended users. The results show that when looking for 
potential dates, males tend to be focused on their own interest and oblivi¬ 
ous towards their attractiveness to potential dates, while females are more 
conscientious to their own attractiveness to and interest from the other side 
on the line. 


The rest of the paper is organized as follows. Section [2] describes the related 
work on reciprocal recommendation as well as reciprocal relation prediction in 
online social networks. Section [3] presents the reciprocal recommendation algo¬ 
rithms we proposed. Description and characteristics of our dataset are provided 
in Section 01 Section [S] presents the performance of our proposed algorithms. 
Finally, we conclude our paper in Section [51 
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2 Related Work 

A few studies on the analysis of user behavior of online dating sites have provided 
valuable guidelines to design recommendation system for online dating. In [3], the 
authors analyze how user’s sending and replying behavior correlate with several 
important user attributes, such as age, income, education level, and number 
of photos, etc., and how much a user’s actual preference deviates from his/her 
stated preference. The findings also correspond to the result of m that the 
recommendation system built on user’s implicit preference outperforms that built 
on user’s explicit preference. 

There exists research that aims to predict user reciprocity in various online 
social networks. In a machine learning based approach is proposed to find 
plausible candidates of business partners using firm profiles and transactional re¬ 
lations between them. The authors of [13] propose a Triad Factor Graph model 
to deal with two-way relation prediction in online social networks. In |5], both 
user-based and graph-based features are applied in a machine learning framework 
to predict user replying behavior in online dating networks. A new collaborative 
filtering approach called CLiMF m is proposed to learn latent factors of users 
and items and improves the performance of top-k recommendations for recom¬ 
mending friends or trustees on Epinions and Tuenti. Further, they improve the 
algorithm by optimizing Expected Reciprocal Rank, an evaluation metric that 
generalizes reciprocal rank in order to incorporate user feedback with multiple 
levels of relevance in m- 

There have been recently several studies on the people to people recommenda¬ 
tion for online social networks. Both content based algorithm and collaborative 
filtering method are applied to recommend users to follow in Twitter m- A 
LDA-based method is employed in |12j to discover communities in Twitter-style 
online social networks, and matrix factorization are applied on each community 
to provide recommendations. 

For online dating recommendations, the authors in [T] and |5] analyzed the 
characteristics of reciprocal recommendations in detail. In particular, [T] consid¬ 
ers both local utility (users’ mutual preference) and global utility (overall bipar¬ 
tite network), and proposes a generalized reciprocal recommendation framework 
for both online dating sites and online recruiting sites. A content based recip¬ 
rocal algorithm (RECON) proposed in [5] learns the preference of both sides of 
users and defines a new evaluation metric (success rate) to evaluate the perfor¬ 
mance of their algorithm. In their following work |1()I9| . RECON is extended to 
consider both positive and negative preference, and collaborative filtering is ap¬ 
plied with stochastic matching algorithm. In |7], a hybrid collaborative filtering 
based algorithm that takes reciprocal links into account is proposed and shown 
to have good performance in recommending both initial and reciprocal contacts. 
A recent study of m finds that users with similar profiles like and dislike similar 
people, and are liked and disliked by similar people. This hypothesis is used to 
build a content-collaborative reciprocal recommender, which is evaluated on a 
popular online dating dataset. In [5], collaborative filtering algorithms are used 
to capture the reciprocity in people to people recommendation. The authors 
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in [3] propose a two-side matching framework for online dating recommenda¬ 
tions and design a Latent Dirichlet Allocation (LDA) model to learn the user 
preferences from the observed user messaging behavior and user profile features. 

The studies most relevant to our online dating recommendation problem are 
[5] and [7], in which a content-based algorithm (RECON) and a collaborative 
filtering-based algorithm (HCF) was presented and shown to outperform other 
approaches. In this paper we will compare the performance of our proposed 
algorithms with these two algorithms. 

3 Recommendation System 

In this section, we propose a generalized reciprocal recommendation system that 
aims to match people with mutual interest in each other. We also introduce two 
previously proposed approaches, namely, a content-based algorithm RECON [S] 
and a hybrid collaborative filtering algorithm (HCF) [7] for comparison. 

3.1 System Design 

The success of a reciprocal recommendation system lies in its ability to recom¬ 
mend users with whom the service user has mutual interest and thus they are 
likely to communicate with each other. The interaction records between a pair 
of users are a good indicator of actual interest and attractiveness between the 
sender and receiver. If a recommended user matches the service user’s interest, 
the service user will be more likely to approach the recommended user. Also, 
if the service user’s attractiveness is compatible with the recommended user’s 
interest, the recommended user will be more likely to reply to the service user 
when contacted. 



— ^ Initial Contact 


Reply 


Fig. 1. Example of an online dating recommendation problem 


Figure [T] depicts an example of an online dating network. Based on user at¬ 
tributes and their communication traces, our goal is to match users with mutual 
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interest in each other, for example, M3 and F3, M4 and F4. In the following 
we will describe how to measure the similarity between a pair of users in terms 
of their dating interest and attractiveness, and how to construct various recom¬ 
mendation algorithms based on these similarity measures. 

Our reciprocal recommendation system is comprised of the following four 
major components. 

Extracting User Based Features: When a user registers with the online 
dating site, he/she needs to provide a variety of profile information including 
the user’s gender, age, current location (city level), home town location, height, 
weight, body type, blood type, occupation, income range, education level, reli¬ 
gion, astrological sign, marriage and children status, photos, home ownership, 
car ownership, interests, smoking and drinking behavior. Most of these attributes 
are categorical features, except age, height, weight and number of photos. 


Algorithm 1 Reciprocal Score(x, y) 

Input: service user x, and recommended user y 
Output: reciprocal score 

begin 

/* initialize compatible scores s*/ 
s{x,y) = 0.0 
s{y,x) = 0.0 

/* compute compatible scores for both x and y * / 
foreach u in Neighbor\{y)-. 

s(x,y) = s(x,y) + Similarityi {x, u) 

end for 

foreach v in Neighbor 2 {x)\ 

s{y,x) = s{y,x) + Similarity 2 {y, v) 

end for 

/* normalize compatible scores for both x and y * j 
if \Neighbori{y)\ > 0 

sir 7/i = __ 

\ ^y) \N eighborx{y)\ 

if \Neighhor 2 {x)\ > 0 

a!,, ri = __ 

\N eighbor2{x)\ 

/* compute reciprocal score */ 
if s{x, y) > 0 and s(y, x) > 0 
return 

else 

return 0.0 

end 


Extracting Graph Based Features: The online dating site we study is for 
heterosexual dating and only allows communications between users of opposite 
sex. The communication traces between users can be modeled as a bipartite net- 
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work between males and females. A collection of graph-based similarity features 
are extracted from the bipartite graph that represent a user’s active level in 
dating and attractiveness. The detailed dehnitions of these graph based features 
are provided in Section [3.21 

Compute Reciprocal Scores: Based on both user-based and graph-based fea¬ 
tures, we use the reciprocal score to measure the mutual interest and attractive¬ 
ness between a pair of potential dates as described in Algorithm [T] 

In Algorithm [TJ Neighborli^ and Neighbor2{) represent the neighbor set 
of a user with different directions in the bipartite network, and their formal 
definitions are given in equations and (O. Similarityi{, ) and Similarity2{,) 
are customizable functions measuring the similarity between two users. We will 
discuss these functions in the following subsection. Given Neighbori, Neighbor2, 
Similarityi and Similarity2, s{x,y) measures the similarity between user x 
and user y’s neighbors, while s{y, x) measures the similarity between user y and 
user x’s neighbors. After computing s(x,y) and s{y,x), the reciprocal score is 
computed as the harmonic mean of these two similarity scores. 

Generate Recommended User List: For a given service user, a recommen¬ 
dation list will be generated by ranking these reciprocal scores. We will present 
the top-K users in the list to the service user. Note that the reciprocal score 
may not be symmetric if Neighbori and Neighbor2 are set as different functions 
or Similarityi and Similarity2 are set as different functions. This is different 
from the case in RECON where there is a unique reciprocal score for any pair 
of users. 

3.2 Similarity Functions 

Content-based Similarity Functions In content-based recommendation sys¬ 
tem, every recommended user can be represented by a feature vector or an at¬ 
tribute profile. These features can be numeric or nominal values representing 
some aspect of the user, such as age, height, income, and etc. Let denote the 
set of known attributes (age, height, income, education level, etc) of user x, i.e.. 

Ax = {a : a is a known attribute of user x} 

We denote the set of user-based attributes of user x as 

Ux = {< : for a G A^}, (1) 

where is the value of attribute a of user x. 

The first similarity measure based on user attributes follows the work of 
RECON [5], where the values of the numeric attributes (e.g., age, height, and 
etc) are grouped into ranged nominal values. Eor each service user, RECON 
builds his/her preference model by constructing the distribution of the receivers’ 
user attributes. How much a service user is interested in a recommended user 
is then measured by comparing the attributes of the recommended user with 
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the preference model of the service user. This is equivalent to computing the 
similarity between two users as follows: 


content-similarity“(x, y) 


SaGAj,nAj, V) 

I Cl I 


( 2 ) 


where 


Pa{x,y) 


1 if 

0, otherwise. 


(3) 


In RECON, numerical values are converted into categorical values, for ex¬ 
ample, users are divided into age groups 20-24, 25-29, 30-34, etc. Note that this 
method does not capture the numerical attribute information at the boundaries 
of continuous ranges. For example, a user of age 25 will not be considered to be 
similar to a user of age 24 as they fall into different age ranges. To avoid the 
loss of information, we modify the similarity measure defined in equation ([2]) as 
follows and use it as our second similarity measure, i.e.. 


content-similarity^ (x, y) 


^aGAa;nAy Qa{x, y) 

\Ax C A.y I 


(4) 


where Qa{x,y) = Pa(x,y) for the nominal attributes as before; for numeric 
attributes, we define 


Qa{x,y) 



(5) 


where v* = max^^^j |u* — vl\ represents the maximum absolute difference for 
attribute a among all users. This new similarity measure results in a value be¬ 
tween 0 when the attributes of the two users have the maximum difference (i.e., 
\Pa ~ xi\ = and 1 when the attributes of the two users are the same (i.e., 
u* = u^). For the above example, the age similarity between a 24-year-old user 
and a 25-year-old user will be 48/49 (very close to 1), where 49 is the max dif¬ 
ference in user ages. It is clear that this new measure can better capture the 
similarity of numerical attributes between two users than the measure defined 
in equation ([2]), and as will be shown in Section [5j results in better performance 
in generating the recommendation list. 

The above two similarity measures are based on user attributes, and can be 
used to construct variations of content-based recommendation algorithms. 


Graph-based Similarity Functions Based on the message traces between 
users, we define the following two graph-based measures to capture the user’s 
active level and attractiveness: 

Se{x) = {y : X has sent a messages to y} (6) 

Re{x) = {y X has received a message from y} (7) 

where Se{x) is defined as the set of out-neighbors of x and its cardinality reflects 
the activeness of user x; Re{x) is dehned as the set of in-neighbors of x and its 
cardinality reflects the attractiveness of user x. 
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Based on the graph-based measures defined in equations and © , we 
introduce the following two similarity functions: 

— Interest similarity: Consider two users of the same gender, x and y. If they 
both contact a same user, it shows that they share common interest in look¬ 
ing for potential dates. The fraction of users who received messages from 
both X and y among all users who received messages from either x or y 
serves as a measure of the similarity between the dating interest of the two 
users, i.e., 

. , IS'efa:) n5'e(?/)| 

mtere8t-s,m,lat,ty(i, y) = ^ (8) 

Note that we adopt the Jaccard Coefficient in the interest similarity measure 
as the number of shared receivers from a pair of users is typically far outnum¬ 
bered by the total number of receivers from the two users. This will become 
clear based on the degree distribution of users shown in Figured and weight 
distribution of the projection network shown in Figure [5] from the dataset 
description in Section U] For the example shown in Figure [U together Ml 
and M2 contacted three different females among which F2 received messages 
from both of them. The interest-similarity between Ml and M2 is thus 1/3. 

— Attractiveness similarity: Consider two users of the same gender, x and y. 
If they both receive messages from a same user, it shows that they share 
common attractiveness to potential dates. The fraction of users who sent 
messages to both x and y among all users who sent messages to either x or 
y serves as a measure of the similarity between the attractiveness of the two 
users, i.e.. 


attractiveness-similarity(a:, y) 


|i?e(a;) C Re{y)\ 
|i?e(a;) U Re{y)\ 


(9) 


By the same token we adopt the Jaccard Coefficient in the attractiveness 
similarity measure. For the example shown in Figure [TJ both FI and F2 
received messages from Ml and M3 while FI also received messages from 
M2. The attractiveness similarity between FI and F2 is thus 2/3. 


3.3 Recommendation Algorithms 

Based on these four similarity functions, we construct the following two content 
based algorithms and four collaborative filtering algorithms. 

The content based algorithms are constructed based on the similarity of user 
attributes, including: 

CBl (RECON): Both Neighbor I and Neighbor 2 in Algorithm [T] are set 
as out-neighbors Se{), and both Similarityi and Similarity2 are computed 
using content similarity defined in equation ([2]). This algorithm is the same 
as RECON 0. 




10 


Peng Xia, Benyuan Liu, Yizhou Sun, and Cindy Chen 


— CB 2 : Both Neighbori and Neighbor2 are set as out-neighbors Se{), and 
both Similarityi and Similarity2 are computed using content similarity 
function defined in equation (|4|), where we do not convert numeric attributes 
into nominal attributes. 

Collaborative filtering-based algorithms make use of the communication his¬ 
tory of the service user as well as the decisions made by users with similar interest 
or attractiveness to help make recommendations. Based on different combina¬ 
tions of users’ dating interest and attractiveness, we construct the following four 
collaborative filtering-based recommendation algorithms: 

— CFl: Both Neighbori and Neighbor2 are set as out-neighbors Se{), and 
both Similarityi and Similarity2 are computed using attractiveness simi¬ 
larity defined in equation ([HI). Therefore, for Algorithm [T] we have 



k^Se{x) 


In this case, s{x,y) sums up the attractiveness similarity (i.e., contacted by 
same users) between service user x and users who have been contacted by 
user y, capturing the attractiveness of the service user x to user y. Simi¬ 
larly, s{y,x) captures the attractiveness of user y to service user x. Putting 
these two factors together, this algorithm captures the mutual attractive¬ 
ness between the service user and recommended users. An example of CFl 
algorithm is shown in Figure Eal 

— CF 2 : Both Neighbori and Neighbor2 are set as in-neighbors i?e(). Both 
Similarityi and Similarity2 are computed using interest similarity defined 
in equation ([8|). In this case, s(x,y) sums up the interest similarity between 
service user x and users who have contacted user y, capturing the interest 
from user x to y. Similarly, s{y,x) captures the interest from user y to x. 
Together, this algorithm captures the mutual interest between the service 
user and recommended users. An example of CF2 algorithm is shown in 
Figure [IB 

— CF 3 : Neighbori is set as out-neighbors Se{), while Neighbor2 is set as 
out-neighbors i?e(). Similarityi is computed using attractiveness similarity 
defined in equation and Similarity2 is computed using interest sim¬ 
ilarity defined in equation ([5]). This algorithm captures the interest from 
recommended users in the service user and the attractiveness of the service 
user to recommended users. An example of CF3 algorithm is shown in Figure 
[ 5 al 

— CF 4 : Neighbori is set as in-neighbors Re{) , while Neighbor2 is set as out- 
neighbors Se{). Similarityi is computed using interest similarity defined in 
equation ([S]), and Similarity2 is computed using attractiveness similarity 
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defined in equation ([S]). This algorithm captures the attractiveness of rec¬ 
ommended users to the service user and the interest of the service user in 
recommended users. An example of CF4 algorithm is shown in Figure l3bl 



Fig. 2. Example of (a)CFl and (b)CF2 algorithms. 


Figure [2] illustrates the mechanism of CFl and CF2 algorithms. In the exam¬ 
ple of CFl shown in Figured^ service user x sent messages to users FI, F2 and 
F3. These out-neighbors of service user x share similar attractiveness with user 
j/, i.e., user y and FI, F2, F3 received messages from at least one common user. 
Also, user y sent messages to users Ml and M2 who share similar attractiveness 
of user X. Therefore, CFl captures the mutual attractiveness between the service 
user and recommended users. If the reciprocate score between x and y ranks in 
the top-K position, user y will be included in the recommendation list for service 
user X. Figure [13 shows an example of CF2, which captures the mutual interest 
between the service user x and user y, i.e., user x's interest in user y and user 
y’s interest in service user x. Examples of CF3 and CF4 illustrated in Figure [3] 
can be interpreted in a similar way. 

In addition to the above algorithms, we also implement the content-based 
algorithm (RECON) proposed in [5] and the hybrid collaborative filtering algo¬ 
rithm (HCE) proposed in [7]. In particular, RECON corresponds to our CBl 
algorithm, where the Neighbori and Neighbor 2 are set as Se{) function, and 
Similarityi and Similarity 2 are computed based on equation HCE extends 
the baseline collaborative filtering approach by considering both initial and re¬ 
ciprocal contacts to compute the similarity between two users, where reciprocal 
links are given higher weight than single direction contacts. These two algorithms 
are most related to our study and have been shown to outperform many other 
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Fig. 3. Example of (a)CF3 and (b)CF4 algorithms. 


approaches. We will compare the performance of our proposed algorithms with 
these two algorithms in Section [5] 

4 Dataset Description 

4.1 Dataset Overview 

The dataset used in our study is obtained through a collaboration with baihe.com, 
one of the major online dating sites in China. Our dataset includes the profile 
information of 200,000 users uniformly sampled from users registered in Novem¬ 
ber of 2011. Of the 200,000 sampled users, 139,482 are males and 60,518 are 
females, constituting 69.7% and 30.3% of the total number of sampled users re¬ 
spectively. For each user, we have his/her message sending and receiving traces 
(who contacted whom at what time) in the online dating site and the profile in¬ 
formation of the users that he or she has communicated with from the date that 
the account was created until the end of January 2012. Note that the site is for 
heterosexual dating and only allows communications between users of opposite 
sex. 

After a user creates an account on the online dating site, he/she can search 
for potential dates based on information within the profiles provided by the other 
users including user location, age, etc. Once a potential date has been discovered, 
the user then sends a message to him/her, which may or may not be replied by 
the recipient. In this paper we focus on the prediction of whether a user will 
reply to initial messages sent by other users. Subsequent interactions between 
the same pair of users do not represent a new sender-receiver pair and can not 
be used as the only indicator for continuing relationship as users may choose to 
go off-line from the site and communicate via other channels (e.g., email, phone 
or meet in person). 
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Table 1. Dataset Description 


Type 

Initial contact 

links 

Reciprocal links 
(Reply rate) 

Male to Sample Female 
Female to Sample Male 

1,586,059 

328,645 

150,917 (9.5%) 
58,946 (17.9%) 


Since we only have eight full weeks’ worth of online dating interaction records 
for our sample users, we will consider the activities of each user during the first 
eight weeks of his/her membership. Table [I] describes the characteristics of the 
dataset. More detailed description and analysis of the dataset can be found in 
our recent work m- 



10 100 1000 10000 100000 
number of messages 

(a) 



10 100 1000 10000 
number of messages 


(b) 


Fig. 4. (a) CCDF of the number of messages a user sent out during the first eight 
weeks of his/her membership, (b) CCDF of the number of messages a user received 
during the first eight weeks of his/her membership. 


For both males and females, we obtain the distribution of the number of 
messages sent by each user per week given that a user sends at least one mes¬ 
sage during the week, and plot its complementary cumulative density function 
(CCDF) in Figure | ^a)[ We observe that the distributions exhibit heavy tails. 
Most users only sent out a small number of messages: 94.6% of males and 96.5% 
of females sent out less than 100 messages during the first eight weeks of their 
membership. On the other hand, there are small fractions of users that sent out 
a large number of messages. According to the online dating site, most of these 
highly active users are likely to be fake identities created by spammers and their 
accounts have been quickly removed from the site. 

The distribution of number of messages received by a user is plotted in Figure 
IIS] A female is likely to receive more messages than a male. Most users only 
received a small number of messages: 99.3% of males and 90.1% of females re¬ 
ceived less than 100 messages during the first eight weeks of their membership. 
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On average, a male received 7 messages while a female received 35 messages 
during the first eight weeks. 



Fig. 5. (a) male sending projection network and (b) female receiving projection network 
corresponding to the example shown in Figure [T] 


4.2 Projection Network Characteristics 

Based on user communication traces, we construct several projection networks 
for each gender and direction of communications (sending or receiving). The 
sending projection network is constructed by adding an edge between two users 
who have sent messages to at least one common receiver, while the receiving 
projection network is constructed by add an edge between two users who received 
messages from at least one common sender. The weight of each edge in a sending 
or receiving projection network denotes the number of common receivers or 
senders between the two nodes, respectively. Figure [S] illustrates the sending 
projection network of male users and receiving projection network of female 
users corresponding to the example shown in Figure [T] 


Table 2. Number of nodes and edges in projection networks. 


Network Type 

Gender 

Nodes 

Edges 

Sending Projection Network 

Male to Female 

75,379 

7,716,078 

Female to Male 

28,550 

1,025,738 

Receiving Projection Network 

Male from Female 

43,420 

22,603,491 

Female from Male 

45,214 

18,858,211 


Table [5] describes several important network measurements of the sending 
and receiving projection networks for male and female users. 





















Reciprocal Recommendation System for Online Dating 


15 


Figure [B] shows the node degree and edge weight distributions for the sending 
projection network of both male and female users. The node degree in a sending 
projection network represents the number of other users with whom the user 
share some degree of similar interest. We observe that a male shares similar 
interest with a larger number of peers than a female. The median degrees for 
male and female users are 89 and 33, respectively. We also observe that most 
edges in the sending projection networks have a low weight (i.e., most pairs of 
users contact very few common receivers), in particular, 73.1% and 76.8% of the 
edges in the male and female sending projection networks have a weight of 1. 
Also, the CCDF of edge weight distribution of females lies above that of males, 
indicating that females tend to share more common receivers than males. 




Fig. 6. CCDF for (a) degree and (b) weight distribution for the sending projection 
networks. 


Figure [7] plots the distributions of node degree and edge weight of the receiv¬ 
ing projection network for both male and female users. The median degrees for 
male and female users are 283 and 551, respectively. Most of the edges have a 
low weight (i.e., most pairs of users are contacted by very few common senders), 
in particular, 85.5% and 70.6% of the edges in the male and female receiving 
projection networks have a weight of 1. 


5 Evaluation 

For a given service users in our test set, we rank the recommended users by 
comparing their reciprocal scores, and recommend the top-K users in the list. 
We evaluate the performance of each algorithm by comparing the top-K users 
in the recommended list with the receivers contacted by the service user in test 
set. 
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Fig. 7. CCDF for (a) degree and (b) weight distribution for the receiving projection 
networks. 



5.1 Experiment Setup 

Most of the active users in our dataset are newly registered users. They are 
usually very active in looking for a potential date in the first one or two weeks 
after registration [5]. We select the user interactions within 10 days from user 
registration time as the training data, and interactions in the remaining time as 
the test set. 

We filter the service users by selecting users who have sent or replied at least 
5 messages in the training period, and use the interactions between these users in 
the test period as the test set. For the training set, we count all the interactions 
initiated, received or replied to by the selected users in the training period, and 
use these interactions to train our recommendation system. Table [3] summarizes 
the experiment dataset. 


Table 3. Experiment Dataset 


Male 

Female 

# of Messages 
in Training Set 

# of Messages 
in Test Set 

24,602 

8,250 

730,110 

270,294 


For RECON (CBl) and CB2, we manually pick 20 features over all 39 fea¬ 
tures. These selected features include age, height, weight, city, education level, 
income, house status, marriage status, children status, physical looking, car sta¬ 
tus, number of photos, smoking habit, drinking habit, marriage status, parents 
status, children plan, dating method, and wedding plan. Among these features, 
age, height, weight, and number of photos are treated as numeric values. For 
HCF, we performed several experiments to get the optimal weight parameter s 
in the computation of the success score between two users [?]■ 











Reciprocal Recommendation System for Online Dating 


17 


5.2 Evaluation Metrics 

For a given service user, we define three set of users: T as the set of users we have 
recommended to the service user, I as the collections of users who have been 
contacted by the service user and R as the set of users who have been contacted 
by the service user and replied to the service user in the test set. We dehne the 
following two different evaluation metrics: 

1/ n T| 1/ n T\ 

I-Precision = ——, I-Recall = —|yj— , (10) 

and 

R-Precision = ——-— R-Recall = - —j—^(11) 

D I 1^1 

where I-Precision and R-Precision measure the ratio of users in the recommen¬ 
dation list who have been contacted by or exchanged messages with the service 
user, respectively. I-Recall and R-Recall measure the ratio of users who have 
been contacted by or exchanged messages with the service user in the list of 
recommended users. From another perspective, I-Precision and I-Recall mea¬ 
sure an algorithm’s performance in recommending users that the service user is 
interested in and thus likely to contact, and R-Precision and R-Recall measure 
an algorithm’s performance in recommending users who have mutual interest 
with the service user and are thus likely to reciprocate when contacted. 


Content Based Algorithm for Male Users 


m 


Fig. 8. I-Precision and I-Recall of content-based algorithms for male users 


Content Based Algorithm for Male Users 

cBi 

CB2 



5 10 20 40 60 80 100 
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5.3 Evaluation Results 

In this subsection, we apply our recommendation algorithms on the experiment 
datasets, and compare the performance with RECON [S] and HCF [7]. We first 
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(a) 


(b) 


Fig. 9. 1-Precision and I-Recall of content-based algorithms for female users 


report the performance of recommending users that the service user will contact, 
and then evaluate the performance of recommending users who will reciprocate 
when contacted by the service user. 



Fig. 10. 1-Precision and 1-Recall of collaborative filtering algorithms for male users 


I-Precision and I-Recall We first examine the performance of these algo¬ 
rithms in recommending users whom the service user is interested in and thus 
likely to contact. 

Figures [S] and ini show the performance of the two content-based algorithms, 
namely, CBl(RECON) and CB2. We observe that by preserving the values of 
numeric attributes in the similarity measure, CB2 significantly outperforms CBl 
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Fig. 11. I-Precision and I-Recall of collaborative filtering algorithms for female users 


in both precision and recall. The improvement is more pronounced for females 
than for males. 



Fig. 12. R-Precision and R-Recall of content-based algorithms for male users. 


Figures [To] and [TT] show the I-Precision and I-Recall of the collaborative fil¬ 
tering based recommendation algorithms for male and female users, respectively. 
We observe that for both male and female users, the four algorithms proposed in 
this paper, CFl to CF4, significantly outperform previously proposed HCF algo¬ 
rithm. For male users, while there are some difference in the performance of CFl, 
CF2, and CF3, the difference is rather small when compared with CF4, which is 
much more effective in attracting the service user to contact the recommended 
users. However, for female users, the four algorithms (CFl to CF4) all perform 
similarly. There is no algorithm significantly outperforming others. Note that the 
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CF4 algorithm captures the interest of the service user in recommended users 
as well as the attractiveness of the recommended users to the service user. The 
results indicate that when it comes to looking for potential dates, males tend to 
be more focused on their own interest and oblivious towards their attractiveness 
to potential dates, while females do not show such behavior. 



Top-K 


Top-K 


(b) 


Fig. 13. R-Precision and R-Recall of content-based algorithms for female users. 


R-Precision and R-Recall We now examine the performance of these algo¬ 
rithms in recommending users who will be contacted by and exchange messages 
with the service user. 

Figures [12] and [T3| show the R-Precision and R-Recall of the two content- 
based algorithms for male and female users, respectively. Similar to I-Precision 
and I-Recall, CB2 performs much better than CBl(RECON), as CB2 does 
not convert numeric attributes into nominal attributes and thus does not loss 
information of these numeric attributes. 

The performance of the collaborative filtering-based algorithms for male and 
female users are shown in Figures [HI and [TSl respectively. The algorithms pro¬ 
posed in our paper (CF1-CF4) still achieve much better results than HCF. For 
male users, while CF4 still outperforms the other algorithms, the difference is not 
as pronounced as for I-Precision and I-Recall measures. For female users, CFl 
and CF2 show very similar behavior. Recall that CF1/CF2 captures the mutual 
interest/attractiveness between the service user and recommended users. This 
shows that learning the mutual interest and mutual attractiveness between two 
users have similar effects for recommending potential dates for females. Unlike 
for male users, CF4 does not outperform other algorithms for female users. On 
the contrary, when the recommendation list (K) becomes large, CF3 starts to 
outperform the other algorithms. Recall that CF3 is symmetric to CF4, captur¬ 
ing the interest from recommended users in the service user and the attractive 
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of the service user to the recommended users. The results indicate that when 
females look for potential dates, they are more conscientious to their own at¬ 
tractiveness to the other side of the line and the other sides’ interest in them. 



(a) (b) 

Fig. 14. R-Precision and R-Recall of collaborative filtering algorithms for male users. 



(a) (b) 

Fig. 15. R-Precision and R-Recall of collaborative filtering algorithms for female 
users. 


5.4 Ranking Effectiveness 

In addition to precision and recall, the relative positions of relevant recommen¬ 
dations are also an important measure for a recommendation system [19) . Rel¬ 
evant recommendations are defined as users in the recommendation list who 
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Top-K 

(a) 


Top-K 

(b) 


Fig. 16. Average effective recommendation position of proposed recommendation al¬ 
gorithms for (a) male users and (b) female users. 


have actually exchanged messages with the service user, i.e., the service user has 
followed the recommendation by contacting the recommended user who in turn 
has replied to the service user. Since a user usually looks at the recommendation 
list from top to bottom, a recommendation system ranking the relevant recom¬ 
mendations in top positions should be considered better than those with similar 
performance in precision and recall but ranking the relevant recommendations 
in lower positions. 



Fig. 17. (a) Age distribution of all users, (b) Age distribution of messages sent. 


Figure [1^] plots the average positions of the relevant recommendations in 
the recommendation list (normalized by the size of recommendation list). All 
of these algorithms rank the effectively recommended users in the top 30% to 
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50% of the recommendation list except for CF3 for female users which ranks the 
relevant recommendations around the halfway of the recommendation list. Note 
that we only present the reciprocal ranking effectiveness of CF1-CF4 algorithms 
as the other algorithms (HCF, CBl and CB2) perform much worse than these 
algorithms. 


5.5 Discussions 

To illustrate the relatively poor performance of the content-based algorithms 
when compared with the collaborative filtering-based approaches, we examine 
the effectiveness of using a user’s attributes to determine whether another user 
would send a message to him or her. 



education level education level 



Fig. 18. (a) Education distribution of all users, (b) Education Status distribution of 
messages sent. 


We plot several attribute distributions of all users as well as those of users 
who received messages for the age (Figure [T71). education level (Figure [TH]), and 
marriage status (Figure IT^ attributes. We observe that for these attributes, the 
distributions for message receivers are quite similar to those for all users with 
small Bhattacharyya distances, indicating that these attributes are not very 
effective in making recommendations. Specifically, the Bhattacharyya distance 
between the two age distributions is 0.122 for males, and 0.064 for females. The 
Bhattacharyya distance between the two education distributions is 0.155 for 
males and 0.011 for females. For marriage status, the Bhattacharyya distance 
is blow 0.032 for both males and females. We also examined other attributes, 
which show small Bhattacharyya distances too. For users in the age range of 
20-30, junior college or bachelor degrees, or single marriage status, it is difficult 
to distinguish them as these users constitute the majority of the population. 
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(a) (b) 


Fig. 19. (a) Marriage distribution of all users, (b) Marriage Status distribution of 
messages sent. 


6 Conclusions 

Matching users with mutual interest in each other is an important task for on¬ 
line dating sites. In this paper, we propose a set of similarity based reciprocal 
recommendation algorithms for online dating. We introduce several similarity 
messures that characterize the attractiveness and interest between two users, 
and select most compatible users for recommendations. We evaluate the perfor¬ 
mance of our proposed algorithms on a large dataset obtained from a major on¬ 
line dating site in China. Our results show that the collaborative filtering-based 
algorithms achieve much better performance than content-based algorithms in 
both precision and recall, and both significantly outperform previously proposed 
approaches. Our results also show that male and female users behave differently 
when it comes to looking for potential dates. In particular, males tend to be 
focused on their own interest and oblivious towards their attractiveness to po¬ 
tential dates, while females are more conscientious to their own attractiveness 
to the other side on the line. 
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